1. Field of the Invention
The present invention relates to an image-capturing device, user interface and method for selective color balance adjustment of an image.
2. Description of the Related Art
In photography and image processing, color balancing is the global adjustment of color intensities in order to adjust specific colors, particularly neutral colors, correctly. Color balance changes the overall mixture of colors in an image for color correction. Color balance is a generalized version of white balance used to get colors other than neutrals to appear accurate or aesthetically pleasing.
Image data acquired by sensors—either film or electronic image sensors—may be transformed from the acquired values to new values that are appropriate for color reproduction or display. Several aspects of the acquisition and display process make such color correction essential—including the fact that the acquisition sensors do not match the sensors in the human eye, that the properties of the display medium must be accounted for, and that the ambient viewing conditions of the acquisition differ from the display viewing conditions.
Human visual perception works well in achieving color balance by perceptual and cognitive mechanisms. However, image-capturing devices such as digital cameras do not perform color balance automatically. There are methods to automatically white balance (AWB) an image by using active lighting, as described in the Dalton et al. patent (U.S. Pat. No. 7,394,488), issued on Jul. 1, 2008. Another method for AWB of an image includes inferring image content, e.g. by detecting human facial region, as proposed in the Berge et al. application (US Patent Application Publication No. 2004/0208363), filed on Apr. 21, 2003. Adjusting color balance on a pixel selective basis is proposed by the Hubina et al. patent (U.S. Pat. No. 6,876,384), issued on Apr. 5, 2005.
Automatic color balancing is not always desirable because it is away to standardize output so a theoretical white card in the image would look white under any illuminant. This automatic feature may undermine the artistic intent of the photographer. For example, an intensely red sunset picture should not be color balanced because it may result in a very bland picture by de-saturating color. Although manual color balance exists in more advanced digital cameras, it is limited to adjustment of global illuminant (tungsten, daylight, fluorescent light, overcast sky or, in professional cameras, correlated color temperature adjustment). The Sakaguchi et al. patent (U.S. Pat. No. 7,212,234), issued on May 1, 2007, discloses a manual color balance graphical user interface using a touch screen.
In scenes with multiple illuminant it can be frustrating to capture images of the scenes because of the varying degree in illuminant. For example, a scene illuminated by a dark blue sky and a near horizontal sun light will have multiple illuminants. In particular, the illuminant from the dark blue sky could have a correlated color temperature (CCT) of 20,000 Kelvin (K), while the portion of the scene illuminated by the near horizontal sun light could have a CCT near 2,000 K. If the global color balance is adjusted considering the dark blue sky it would make the portion of the image illuminated by horizontal sun light even yellower. If the global color balance is adjusted considering the yellowish portion of the scene illuminated by the horizontal rays coming from the sun, the dark shadows would look even bluer. Any global color balance using a compromise of CCT between both extremes would make the colors in both regions incompletely corrected. Therefore, there is no ideal global solution for color balance for such a scene.
Conceptually, color balancing consists of two steps: first, determining the illuminant under which an image was captured; and second, scaling the components (e.g., R, G, and B) of the image or otherwise transforming the components so they conform to the viewing illuminant. The color balance operations in popular image editing applications usually operate directly on the red, green, and blue channel pixel values, without respect to any color sensing or reproduction model. A satisfactory image may be produced manually after time-consuming adjustments using image-editing software.
Where there are ways to control color balance pixel by pixel, there does not exist an interface for a user of an image-capturing device to specify which region of the image to color balance and to what degree of color balancing the user would like the image-capturing device to carry out. As a result, photographers rely heavily on post-processing to get desired results. However, if the digital camera shot the image with inappropriate color balance, it is very difficult to correct it regardless of the image editing software. Furthermore, automatic global corrections of complex scenes illuminated by multiple illuminants with very different CCTs tend to produce images that have wrong color balance.
Color balance may refer to correction for differences in the ambient illuminant conditions. However, the algorithms for transforming the data do not always clearly describe the adjustments needed to account for differences between the sensors and the human eye. Hence, it can be difficult to assign color balance to a specific step in the color correction process. Moreover, there can be significant differences in the color-balancing goal. Some applications produce an accurate rendering. In other applications, the goal of color balancing is to produce an aesthetically pleasing rendering. Photographers may deliberately set-up “wrong” CCT or illuminant in the manual color balance to produce an aesthetically pleasant image. For example, a scene with sunset has low CCT but the photographer deliberately sets up a high CCT (or an illuminant with higher CCT in the camera control) so the camera applies a red shift into the image exacerbating the yellowish-reddish look of the scene to convey an even warmer feeling of the scene.
Most digital cameras have a means to select a color correction based on the type of scene illuminant, using either manual illuminant selection, or AWB, or custom white balance. However, such selections are performed globally to the whole image. Setting a button on a camera is a way for the user to indicate to the processor the nature of the scene lighting. Another option on some cameras is a button pressed with the camera pointed at a gray card or other neutral object. This “custom white balance” step captures an image of the ambient light, and this information is helpful in controlling color balance.
Once an image is captured with inadequate or non-desirable color balance it often has saturated color channels. This means that information is lost and there is no way to recover information lost in the capture. This means it is advantageous to get desired color balance at the time an image is captured, rather than edit later on a monitor using image-editing software.
Accordingly, an intuitive method to select different regions of an image and apply varying degrees of manual color balance would be advantageous and would reduce the need for post-processing color balance correction and at the same time would enable multi-illumination color balance adjustment.